User-centric evaluation of recommender systems in social learning platforms: Accuracy is just the tip of the iceberg

Recommender systems provide users with content they might be interested in. Conventionally, recommender systems are evaluated mostly by using prediction accuracy metrics only. But the ultimate goal of a recommender system is to increase user satisfaction. Therefore, evaluations that measure user satisfaction should also be performed before deploying a recommender system in a real target environment. Such evaluations are laborious and complicated compared to the traditional, data-centric evaluations, though. In this study, we carried out a user-centric evaluation of state-of-the-art recommender systems as well as a graph-based approach in the ecologically valid setting of an authentic social learning platform. We also conducted a data-centric evaluation on the same data to investigate the added value of user-centric evaluations and how user satisfaction of a recommender system is related to its performance in terms of accuracy metrics. Our findings suggest that user-centric evaluation results are not necessarily in line with data-centric evaluation results. We conclude that the traditional evaluation of recommender systems in terms of prediction accuracy only does not suffice to judge performance of recommender systems on the user side. Moreover, the user-centric evaluation provides valuable insights in how candidate algorithms perform on each of the five quality metrics for recommendations: usefulness, accuracy, novelty, diversity, and serendipity

Neoadjuvant chemoradiotherapy plus surgery versus active surveillance for oesophageal cancer: A stepped-wedge cluster randomised trial

Background: Neoadjuvant chemoradiotherapy (nCRT) plus surgery is a standard treatment for locally advanced oesophageal cancer. With this treatment, 29% of patients have a pathologically complete response in the resection specimen. This provides the rationale for investigating an active surveillance approach. The aim of this study is to assess the (cost-)effectiveness of active surveillance vs. standard oesophagectomy after nCRT for oesophageal cancer. Methods: This is a phase-III multi-centre, stepped-wedge cluster randomised controlled trial. A total of 300 patients with clinically complete response (cCR, i.e. no local or disseminated disease proven by histology) after nCRT will be randomised to show non-inferiority of active surveillance to standard oesophagectomy (non-inferiority margin 15%, intra-correlation coefficient 0.02, power 80%, 2-sided α 0.05, 12% drop-out). Patients will undergo a first clinical response evaluation (CRE-I) 4-6 weeks after nCRT, consisting of endoscopy with bite-on-bite biopsies of the primary tumour site and other suspected lesions. Clinically complete responders will undergo a second CRE (CRE-II), 6-8 weeks after CRE-I. CRE-II will include 18F-FDG-PET-CT, followed by endoscopy with bite-on-bite biopsies and ultra-endosonography plus fine needle aspiration of suspected lymph nodes and/or PET- positive lesions. Patients with cCR at CRE-II will be assigned to oesophagectomy (first phase) or active surveillance (second phase of the study). The duration of the first phase is determined randomly over the 12 centres, i.e., stepped-wedge cluster design. Patients in the active surveillance arm will undergo diagnostic evaluations similar to CRE-II at 6/9/12/16/20/24/30/36/48 and 60 months after nCRT. In this arm, oesophagectomy will be offered only to patients in whom locoregional regrowth is highly suspected or proven, without distant dissemination. The main study parameter is overall survival; secondary endpoints include percentage of patients who do not undergo surgery, quality of life, clinical irresectability (cT4b) rate, radical resection rate, postoperative complications, progression-free survival, distant dissemination rate, and cost-effectiveness. We hypothesise that active surveillance leads to non-inferior survival, improved quality of life and a reduction in costs, compared to standard oesophagectomy. Discussion: If active surveillance and surgery as needed after nCRT leads to non-inferior survival compared to standard oesophagectomy, this organ-sparing approach can be implemented as a standard of care

RAGE Reusable Game Software Components and Their Integration into Serious Game Engines

This paper presents and validates a methodology for integrating reusable software components in diverse game engines. While conforming to the RAGE com-ponent-based architecture described elsewhere, the paper explains how the interac-tions and data exchange processes between a reusable software component and a game engine should be implemented for procuring seamless integration. To this end, a RAGE-compliant C# software component providing a difficulty adaptation routine was integrated with an exemplary strategic tile-based game “TileZero”. Implementa-tions in MonoGame, Unity and Xamarin, respectively, have demonstrated successful portability of the adaptation component. Also, portability across various delivery platforms (Windows desktop, iOS, Android, Windows Phone) was established. Thereby this study has established the validity of the RAGE architecture and its un-derlying interaction processes for the cross-platform and cross-game engine reuse of software components. The RAGE architecture thereby accommodates the large scale development and application of reusable software components for serious gaming

Automated Adaptation and Assessment in Serious Games: a Portable Tool for Supporting Learning

We introduce the Adaptation and Assessment (TwoA) component, an open-source tool for serious games, capable of adjusting game difficulty to player skill level. Technically, TwoA is compliant with the RAGE (Horizon 2020) game component architecture, which offers seamless portability to a variety of popular game development platforms. Conceptually, TwoA uses a modified version of the Computer Adaptive Practice algorithm. Our version offers two improvements over the original algorithm. First, the TwoA improves balancing of player's motivation and game challenge. Second, TwoA reduces the selection bias that may arise for items of similar difficulty by adopting a fuzzy selection rule. These improvements are validated using multi-agent simulations

Efficient Software Assets for Fostering Learning in Applied Games

Digital game technologies are a promising way to enable training providers to reach other target groups, namely those who are not interested in traditional learning technologies. Theoretically, through using digital game technologies we are able to foster the acquisition of any competence by specifying competency structures, offering adequate problem solving support while maintaining motivation and taking personality into consideration as part of the tailored game experience. In this paper, we illustrate how this is done within the RAGE project, which aims to develop, transform, and enrich advanced technologies into self-contained gaming assets for the leisure games industry to support game studios in developing applied games easier, faster, and more cost effectively. The software assets discussed here represent a modular approach for fostering learning in applied games. These assets address four main pedagogical functions: competency structures (i.e., logical order for learning), motivation, performance support (i.e., guidance to maintain learning), and adaption to the player’s personality

El Obrero : semanario de la asociación protectora de la clase obrera de Pontevedra.: Ano II Número 45 - 10 maio 1891

This deliverable deals with the asset integration methodology employed in the RAGE project. A considerable amount of the effort of the RAGE project has been directed towards addressing the technical challenges of asset integration and interoperability. All RAGE partners engaged in asset development adopted a pragmatic approach to integration and interoperability and where practical adopted the use of Open Standards and Specifications supported by a defined project architecture. Asset developers adopted a consistent approach to development languages. The four RAGE Game development partners each included a number of assets within their development processes and in producing the RAGE pilot Games. Their experience of the asset-based development methodology is detailed in this deliverable. Where potential gaps in the interoperability integration activity were identified, specifically in interoperability between the output of games data and Learning Management systems, the project undertook specific “proof of concept” work to address this. Where proposed activity was identified by the development community or through consultation as being problematic or as a potential barrier to adoption, for example the case of a formal nomenclature, effort was directed to more productive activity. This approach unearthed technical challenges, which were managed by the RAGE project Integration and interoperability group All issues were successfully resolved during the research and development phase of the project. Ultimately only minor integration problems occurred and these are documented within the Game development partners input into this deliverable. The RAGE project’s pragmatic approach to integration and interoperability has been very well received by those engaging with the project also serving to “cross the chasm” from both technical and philosophical perspectives and between academia and Industry. In doing so the project has achieved a number of the key primary objectives. Conceptually, from a technical perspective, the RAGE asset-based approach to Applied Game development has been validated